Observation Error Covariance Matrix Research Articles

Including the Horizontal Observation Error Correlation in the Ensemble Kalman Filter: Idealized Experiments with NICAM-LETKF

Abstract Densely observed remote sensing data such as radars and satellites generally contain significant spatial error correlations. In data assimilation, the observation error covariance matrix is usually assumed to be diagonal, and the dense data are thinned or spatially averaged to compensate for neglecting the spatial observation error correlation. However, in theory, including the spatial observation error correlation in data assimilation can make better use of the dense data. This study performs perfect model observing system simulation experiments (OSSEs) using the nonhydrostatic icosahedral atmospheric model (NICAM) and the local ensemble transform Kalman filter (LETKF) to assess the impact of assimilating horizontally dense and error-correlated observations. The condition number of the observation error covariance matrix, defined as the ratio of the largest to smallest eigenvalues, is important for the numerical stability of the LETKF computation. A large condition number makes it difficult to compute the ensemble transform matrix correctly. Reducing the condition number by reconditioning is found effective for stable computation. The results show that including the horizontal observation error correlation with reconditioning makes the analysis more accurate but requires 6 times more computations than the case with the diagonal observation error covariance matrix. Significance Statement It is important to effectively utilize observations in data assimilation for more accurate weather prediction. Spatially dense observations are known to have an error correlation that is ignored in the data assimilation. This study explores assimilating dense observations by explicitly including observation error correlations with an idealized experiment. The results shows that the analysis is improved by including the observation error correlations. Also, the condition number of the observation error covariance matrix is essential for stable computations.

УЧЕТ СКОРРЕЛИРОВАННОСТИ ОШИБОК СПУТНИКОВЫХ ДАННЫХ НАБЛЮДЕНИЙ AMV В АНСАМБЛЕВОЙ СИСТЕМЕ УСВОЕНИЯ ДАННЫХ

The effect of considering correlated errors in AMV (Atmospheric Motion Vectors) satellite observations of wind in the local ensemble transform Kalman filter data assimilation system is studied. It is customary to use a diagonal covariance matrix of errors in observations taking part in assimilation. When assimilating satellite data with correlated errors, the data are thinned, and the values of diagonal elements of the error covariance matrix are often overestimated. This is accompanied by the loss of useful information about the correlation between the errors. The present study uses a different method: the elements of the error covariance matrix for AMV satellite observations are simulated using a second-order autoregressive function. It is shown that such approach reduces the root-mean-square error in initial data for a numerical weather prediction model, in particular on small scales, and improves the forecast quality. It is found that the application of the non-diagonal AMV observation error covariance matrix increases the accuracy of analysis and forecast fields.

HSC-XXL: Baryon budget of the 136 XXL groups and clusters

Abstract We present our determination of the baryon budget for an X-ray-selected XXL sample of 136 galaxy groups and clusters spanning nearly two orders of magnitude in mass (M500 ∼ 1013–1015 M⊙) and the redshift range 0 ≲ z ≲ 1. Our joint analysis is based on the combination of Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) weak-lensing mass measurements, XXL X-ray gas mass measurements, and HSC and Sloan Digital Sky Survey multiband photometry. We carry out a Bayesian analysis of multivariate mass-scaling relations of gas mass, galaxy stellar mass, stellar mass of brightest cluster galaxies (BCGs), and soft-band X-ray luminosity, by taking into account the intrinsic covariance between cluster properties, selection effect, weak-lensing mass calibration, and observational error covariance matrix. The mass-dependent slope of the gas mass–total mass (M500) relation is found to be $1.29_{-0.10}^{+0.16}$, which is steeper than the self-similar prediction of unity, whereas the slope of the stellar mass–total mass relation is shallower than unity; $0.85_{-0.09}^{+0.12}$. The BCG stellar mass weakly depends on cluster mass with a slope of $0.49_{-0.10}^{+0.11}$. The baryon, gas mass, and stellar mass fractions as a function of M500 agree with the results from numerical simulations and previous observations. We successfully constrain the full intrinsic covariance of the baryonic contents. The BCG stellar mass shows the larger intrinsic scatter at a given halo total mass, followed in order by stellar mass and gas mass. We find a significant positive intrinsic correlation coefficient between total (and satellite) stellar mass and BCG stellar mass and no evidence for intrinsic correlation between gas mass and stellar mass. All the baryonic components show no redshift evolution.

Geostationary Hyperspectral Infrared Sounder Channel Selection for Capturing Fast-Changing Atmospheric Information

Various methodologies have been developed for selecting a subset of channels from a hyperspectral infrared (IR) sounder for assimilation. The information entropy iterative method was considered optimal for channel selection. However, this method only considers the decrease in uncertainty in the atmospheric state caused by measurements at a single time, without considering the dynamic effect of measurements over a period of time; therefore, it might not be optimal for hyperspectral IR sounders onboard geosynchronous satellites that mainly aim to observe rapidly changing weather events. An alternative channel selection method is developed by adding an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> index, which reflects the Jacobian variance over time; the adjusted algorithm is ideal for the Geosynchronous Interferometric Infrared Sounder (GIIRS), which is the first high-spectral-resolution advanced IR sounder onboard a geostationary weather satellite. Comparisons between the conventional algorithm (information entropy iterative method) and the adjusted algorithm show that the channels selected from GIIRS by the adjusted algorithm will have larger brightness temperature diurnal variations and better information content than the conventional algorithm, based on the same background error covariance matrix, the observational error covariance matrix, and the channel blacklist. The adjusted algorithm is able to select the channels for monitoring atmospheric temporal variation while retaining the information content from the conventional method. The 1-D variational (1Dvar) retrieval experiment also verifies the superiority of this adjusted algorithm; it indicates that using the channel selected by the adjusted algorithm could enhance the water vapor profile retrieval accuracy, especially for the lower and middle troposphere atmosphere.

Interpreting estimated observation error statistics of weather radar measurements using the ICON-LAM-KENDA system

Abstract. Assimilation of weather radar measurements including radar reflectivity and radial wind data has been operational at the Deutscher Wetterdienst, with a diagonal observation error (OE) covariance matrix. For an implementation of a full OE covariance matrix, the statistics of the OE have to be a priori estimated, for which the Desroziers method has been often used. However, the resulted statistics consists of contributions from different error sources and are difficult to interpret. In this work, we use an approach that is based on samples for truncation error in radar observation space to approximate the representation error due to unresolved scales and processes (RE) and compare its statistics with the OE statistics estimated by the Desroziers method. It is found that the statistics of the RE help the understanding of several important features in the variances and correlation length scales of the OE for both reflectivity and radial wind data and the other error sources from the microphysical scheme, radar observation operator and the superobbing technique may also contribute, for instance, to differences among different elevations and observation types. The statistics presented here can serve as a guideline for selecting which observations are assimilated and for assignment of the OE covariance matrix that can be diagonal or full and correlated.

New bounds on the condition number of the Hessian of the preconditioned variational data assimilation problem

AbstractData assimilation algorithms combine prior and observational information, weighted by their respective uncertainties, to obtain the most likely posterior of a dynamical system. In variational data assimilation the posterior is computed by solving a nonlinear least squares problem. Many numerical weather prediction (NWP) centers use full observation error covariance (OEC) weighting matrices, which can slow convergence of the data assimilation procedure. Previous work revealed the importance of the minimum eigenvalue of the OEC matrix for conditioning and convergence of the unpreconditioned data assimilation problem. In this article we examine the use of correlated OEC matrices in the preconditioned data assimilation problem for the first time. We consider the case where there are more state variables than observations, which is typical for applications with sparse measurements, for example, NWP and remote sensing. We find that similarly to the unpreconditioned problem, the minimum eigenvalue of the OEC matrix appears in new bounds on the condition number of the Hessian of the preconditioned objective function. Numerical experiments reveal that the condition number of the Hessian is minimized when the background and observation lengthscales are equal. This contrasts with the unpreconditioned case, where decreasing the observation error lengthscale always improves conditioning. Conjugate gradient experiments show that in this framework the condition number of the Hessian is a good proxy for convergence. Eigenvalue clustering explains cases where convergence is faster than expected.

Estimation of the error covariance matrix for IASI radiances and its impact on the assimilation of ozone in a chemistry transport model

Abstract. In atmospheric chemistry retrievals and data assimilation systems, observation errors associated with satellite radiances are chosen empirically and generally treated as uncorrelated. In this work, we estimate inter-channel error covariances for the Infrared Atmospheric Sounding Interferometer (IASI) and evaluate their impact on ozone assimilation with the chemistry transport model MOCAGE (Modèle de Chimie Atmosphérique à Grande Echelle). The method used to calculate observation errors is a diagnostic based on the observation and analysis residual statistics already adopted in many numerical weather prediction centres. We used a subset of 280 channels covering the spectral range between 980 and 1100 cm−1 to estimate the observation-error covariance matrix. This spectral range includes ozone-sensitive and atmospheric window channels. We computed hourly 3D-Var analyses and compared the resulting O3 fields against ozonesondes and the measurements provided by the Microwave Limb Sounder (MLS) and by the Ozone Monitoring Instrument (OMI). The results show significant differences between using the estimated error covariance matrix with respect to the empirical diagonal matrix employed in previous studies. The validation of the analyses against independent data reports a significant improvement, especially in the tropical stratosphere. The computational cost has also been reduced when the estimated covariance matrix is employed in the assimilation system, by reducing the number of iterations needed for the minimizer to converge.

Data assimilation with multiple types of observation boreholes via the ensemble Kalman filter embedded within stochastic moment equations

Abstract. We employ an approach based on the ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices corresponding to (i) multi-node monitoring wells equipped with packers (Type A) and (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality, (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality, (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when a simplified flow model is adopted, and (iv) when compared with the standard Monte Carlo-based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

Adaptive Kalman filter-based information fusion in electrical impedance tomography for a two-phase flow

Electrical impedance tomography (EIT) is a severely ill-posed nonlinear inverse problem. In order to obtain solutions with physical meaning, the inverse of the model of measurements requires the combination of information from various sources. This paper proposes a new approach through Kalman filtering for adaptive integration of EIT measurements, Tikhonov regularization and evolution models for the characterization of a two-phase air–water fluid flow. The Tikhonov regularization factor is embedded into the observation error covariance matrix, thus allowing for individual adjustment for each of the regularization equations. The filter outputs for different evolution models—random walk, advective and advective–diffusive—are compared in terms of estimate convergence and physical meaning. With the random walk evolution model the analysis of experimental data shows that the proposed information fusion strategy provides fewer artifacts, enabling a more effective identification of the phase interfaces. When the other two evolution models are incorporated into the Kalman filter and compared with the random walk model, faster and more accurate estimates of the flow are obtained even away from the electrodes, as well as sharper phase interfaces are identified. The results suggest that the reason for this improved performance is the fused information from the upstream–downstream dynamics of the advective and advective–diffusive models with the outer-inner structure influence of measurements.

Increasing the Space–Time Resolution of Mapped Sea Surface Height From Altimetry

AbstractWe present a mapping methodology to improve the resolution of gridded sea surface height (SSH) from multisatellite along‐track measurements, and apply it in the California Current system. This is motivated by the upcoming Surface Water and Ocean Topography (SWOT) satellite mission, which will collect novel high‐resolution wide‐swath SSH measurements. The calibration/validation of SWOT will be in the California Current and requires independent observations of SSH spatial variability. Our objective was to discover how far we could push the space–time resolution of SSH maps based on the existing altimetry constellation. Currently, the best SSH maps for the region are from AVISO's global product based on optimal interpolation (OI). Our methodology is based on 2‐D variational analysis (2DVAR), and we address some limitations of the AVISO OI global parameterization that restrict the resolution. Key improvements are (1) use of a background field incorporating mesoscale information, as opposed to a long‐term mean field; (2) optimized regional parameters; and (3) a representation error term in the observational error covariance matrix to account for the spread of measurements in time. Using a suite of independent remote and in situ observations, we show the 2DVAR method can produce higher‐resolution SSH maps in the California Current region that successfully capture smaller‐scale ocean features than AVISO, at the expense of an increase in noise.

The impact of using reconditioned correlated observation‐error covariance matrices in the Met Office 1D‐Var system

AbstractRecent developments in numerical weather prediction have led to the use of correlated observation‐error covariance (OEC) information in data assimilation and forecasting systems. However, diagnosed OEC matrices are often ill‐conditioned and may cause convergence problems for variational data assimilation procedures. Reconditioning methods are used to improve the conditioning of covariance matrices while retaining correlation information. In this article, we study the impact of using the “ridge regression” method of reconditioning to assimilate Infrared Atmospheric Sounding Interferometer (IASI) observations in the Met Office 1D‐Var system. This is the first systematic investigation of how changing target condition numbers affects convergence of a 1D‐Var routine. This procedure is used for quality control, and to estimate key variables (skin temperature, cloud‐top pressure, cloud fraction) that are not analysed by the main 4D‐Var data assimilation system. Our new results show that the current (uncorrelated) OEC matrix requires more iterations to reach convergence than any choice of correlated OEC matrix studied. This suggests that using a correlated OEC matrix in the 1D‐Var routine would have computational benefits for IASI observations. Using reconditioned correlated OEC matrices also increases the number of observations that pass quality control. However, the impact on skin temperature, cloud fraction, and cloud‐top pressure is less clear. As the reconditioning parameter is increased, differences between retrieved variables for correlated OEC matrices and the operational diagonal OEC matrix reduce. As correlated choices of OEC matrix yield faster convergence, using stricter convergence criteria along with these matrices may increase efficiency and improve quality control.

3D seismic data assimilation to reduce uncertainties in reservoir simulation considering model errors

Decisions in petroleum reservoir management usually involve high level of uncertainties. Therefore, information of different types is used to calibrate reservoir models for the production forecasts and decision analysis. One source of information is 3D seismic, which is highly correlated to petrophysical properties. These properties are a major source of uncertainty. The incorporation of 3D seismic data in flow models is affected by errors caused by discretization, scale differences, seismic modeling uncertainties, seismic propagation related distortions, among others. Nevertheless, these errors are commonly neglected in conventional model calibration workflows. This work treats seismic resolution loss as a form of model error that needs to be considered in the data assimilation process. In our tests, we used the synthetic data from a realistic benchmark case. First, we extended the methodology proposed by Oliver and Alfonzo (2018a) to 3D seismic data assimilation. We focus on the model improvement by estimating a “total” observation error covariance matrix. Furthermore, we reduced the influence of systematic errors by including a simple analytical function in our forward model. The function is defined based on physical premises and the parameter is calibrated in the data assimilation workflow. This procedure increases the dimension of the problem. The error covariance update improved the reservoir volume characterization in all of our tests. Moreover, we show that the update provides a way to improve the determination of the residual weights in the data assimilation problem. These weights are difficult to define in practice and the results were relatively insensitive to the initial values. By using the proposed methodology, we were able to improve the reservoir volume calibration using a relatively low-resolution data. If the correlated errors were neglected, the data assimilation would lead to implausible parameter distributions.

Improving the condition number of estimated covariance matrices

High dimensional error covariance matrices and their inverses are used to weight the contribution of observation and background information in data assimilation procedures. As observation error covariance matrices are often obtained by sampling methods, estimates are often degenerate or ill-conditioned, making it impossible to invert an observation error covariance matrix without the use of techniques to reduce its condition number. In this paper, we present new theory for two existing methods that can be used to ‘recondition’ any covariance matrix: ridge regression and the minimum eigenvalue method. We compare these methods with multiplicative variance inflation, which cannot alter the condition number of a matrix, but is often used to account for neglected correlation information. We investigate the impact of reconditioning on variances and correlations of a general covariance matrix in both a theoretical and practical setting. Improved theoretical understanding provides guidance to users regarding method selection, and choice of target condition number. The new theory shows that, for the same target condition number, both methods increase variances compared to the original matrix, with larger increases for ridge regression than the minimum eigenvalue method. We prove that the ridge regression method strictly decreases the absolute value of off-diagonal correlations. Theoretical comparison of the impact of reconditioning and multiplicative variance inflation on the data assimilation objective function shows that variance inflation alters information across all scales uniformly, whereas reconditioning has a larger effect on scales corresponding to smaller eigenvalues. We then consider two examples: a general correlation function, and an observation error covariance matrix arising from interchannel correlations. The minimum eigenvalue method results in smaller overall changes to the correlation matrix than ridge regression but can increase off-diagonal correlations. Data assimilation experiments reveal that reconditioning corrects spurious noise in the analysis but underestimates the true signal compared to multiplicative variance inflation.

Towards a Data-Derived Observation Error Covariance Matrix for Satellite Measurements

The observation error covariance (R) matrix is a key component in the data assimilation (DA) process for retrieval of atmospheric state parameters (ASPs), also impacting the subsequent numerical weather forecast. However, one commonly used type of R matrix depends on instrument noise, which contravenes reality because the retrieved ASPs would depend on the instrument used. Other types of R matrix rely on the observation operator (H), analyzed state ( x a ), background error covariance (B) matrix or the background state ( x b ), and the selected forecast ensemble. All these dependences reduce the representativeness of the R matrix, since the correctness of H needs verification and no true values exist for x a or x b . As such, a better method to correctly specify the R matrix is needed. Through the physical mechanism occurring between incident radiation and particles in the atmosphere, which complies with the phenomena of energy absorption and emission, correlations among bands or channels in a detected atmospheric radiance spectrum occur. This paper thus proposes a data-derived R matrix based on a large number (N) of detected atmospheric radiance spectra constructed from N real-time measurements, where N real-time measurements can be acquired by staring at some observation location of interest during a short amount of time. This data-derived R matrix for satellite radiance observations does not rely on any assumed quantities and is unambiguous. Technically, recording N real-time measurements is achievable by modifying the trigger configuration of data recording from ground.

Characterizing the Representativity Error of Cloud Profiling Observations for Data Assimilation

AbstractActive observations from profiling instruments such as cloud radar or lidar have proved invaluable in improving our understanding of clouds and their interaction with the atmosphere. However, while they undoubtedly contain a wealth of information about the atmospheric state, the assimilation of such observations into NWP models has so far had mixed success. One reason is the mismatch of scales between the relatively coarse model resolution and the narrow beam‐width of the instruments. To obtain an optimal analysis, this representativity error due to this mismatch in scales must be included within the observation error covariance matrix. In this paper, we demonstrate how a relatively straightforward approach that uses the local variance of the measurement combined with a global estimate of correlation provides a useful estimate of the representativity error. We also show how the method can be used to estimate the correlation in representativity error between observations in both the horizontal and vertical, potentially removing the need to thin the observations, or inflate their observation error. The method is evaluated using 2‐D and 3‐D fields of simulated and observed radar reflectivity before an example application to profiling observations is shown using Cloudsat data.

Considering the impact of observation error correlation in ensemble square-root Kalman filter

Data assimilation has been developed into an effective technology that can utilize a large number of multisource unconventional data. It cannot only provide the initial field for the ocean numerical prediction model, but also construct the ocean reanalysis datasets and provide the design basis for the ocean observation plan. In data assimilation, the estimation of the observation error is of paramount importance, because the quality of the analysis depends on it. In general, the observation error covariance matrix is diagonal or assumed to be diagonal, which means that the observation errors are independent from one another. However, there are indeed correlations in the observation errors. A diagnostic method has been developed, which can estimate a correlated and more accurate observation error covariance matrix. The proposed method combines an ensemble squareroot Kalman filter with the diagnostic method, providing an estimation of the observation error covariance matrix. In order to test the performance of the method, the numerical experiments are performed with the Lorenz 96 model and a Shallow water model. The more accurate observation error covariance matrix can be obtained to use in ensemble square-root Kalman filter by using the new method. We could find using the estimated correlated observation error in the data assimilation improves the analysis.

Assimilation of IASI Ozone‐Sensitive Channels in Preparation for an Enhanced Coupling Between Numerical Weather Prediction and Chemistry Transport Models

AbstractIn this study, IASI ozone‐sensitive channels have been assimilated in 1D‐Var data assimilation combined with realistic ozone background coming from a MOCAGE (Modèle de Chimie Atmosphérique à Grande Echelle) Chemistry Transport Model (CTM) as a first stage of coupling between Numerical Weather Prediction (NWP) and MOCAGE CTM at Météo‐France for global model ARPEGE (Action de Recherche Petite Echelle Grande Echelle). To evaluate the impact of ozone‐sensitive channels on analyses, databases of 161 temperatures, humidity, and ozone radiosondes across the globe during a 1‐year period have been considered. Ozone forecast from MOCAGE CTM was evaluated with radiosondes and used as input for the Radiative Transfer Model (RTM) RTTOV. Statistics of IASI observations minus simulations show that the use of ozone from MOCAGE CTM allows to better simulate IASI ozone‐sensitive channels. The Desroziers method is used to diagnose observation error covariance matrix and estimate realistic ozone observation standard errors taking into account cross‐correlations between IASI channels. The background error covariance matrix for ozone is estimated using radiosondes. A control run assimilating 123 IASI operational channels is compared to an experiment which assimilates, in addition, 15 IASI ozone‐sensitive channels. Results show potential benefits of IASI ozone‐sensitive channels combined with realistic ozone from MOCAGE CTM to improve temperature, humidity and ozone analyses simultaneously. This work is an encouraging first step for the enhancing of the coupling between the global model ARPEGE and MOCAGE CTM.

Efficient big data assimilation through sparse representation: A 3D benchmark case study in petroleum engineering.

Data assimilation is an important discipline in geosciences that aims to combine the information contents from both prior geophysical models and observational data (observations) to obtain improved model estimates. Ensemble-based methods are among the state-of-the-art assimilation algorithms in the data assimilation community. When applying ensemble-based methods to assimilate big geophysical data, substantial computational resources are needed in order to compute and/or store certain quantities (e.g., the Kalman-gain-type matrix), given both big model and data sizes. In addition, uncertainty quantification of observational data, e.g., in terms of estimating the observation error covariance matrix, also becomes computationally challenging, if not infeasible. To tackle the aforementioned challenges in the presence of big data, in a previous study, the authors proposed a wavelet-based sparse representation procedure for 2D seismic data assimilation problems (also known as history matching problems in petroleum engineering). In the current study, we extend the sparse representation procedure to 3D problems, as this is an important step towards real field case studies. To demonstrate the efficiency of the extended sparse representation procedure, we apply an ensemble-based seismic history matching framework with the extended sparse representation procedure to a 3D benchmark case, the Brugge field. In this benchmark case study, the total number of seismic data is in the order of . We show that the wavelet-based sparse representation procedure is extremely efficient in reducing the size of seismic data, while preserving the salient features of seismic data. Moreover, even with a substantial data-size reduction through sparse representation, the ensemble-based seismic history matching framework can still achieve good estimation accuracy.

Improving Particle Filter Performance by Smoothing Observations

AbstractThis article shows that increasing the observation variance at small scales can reduce the ensemble size required to avoid collapse in particle filtering of spatially extended dynamics and improve the resulting uncertainty quantification at large scales. Particle filter weights depend on how well ensemble members agree with observations, and collapse occurs when a few ensemble members receive most of the weight. Collapse causes catastrophic variance underestimation. Increasing small-scale variance in the observation error model reduces the incidence of collapse by de-emphasizing small-scale differences between the ensemble members and the observations. Doing so smooths the posterior mean, though it does not smooth the individual ensemble members. Two options for implementing the proposed observation error model are described. Taking a discretized elliptic differential operator as an observation error covariance matrix provides the desired property of a spectrum that grows in the approach to small scales. This choice also introduces structure exploitable by scalable computation techniques, including multigrid solvers and multiresolution approximations to the corresponding integral operator. Alternatively the observations can be smoothed and then assimilated under the assumption of independent errors, which is equivalent to assuming large errors at small scales. The method is demonstrated on a linear stochastic partial differential equation, where it significantly reduces the occurrence of particle filter collapse while maintaining accuracy. It also improves continuous ranked probability scores by as much as 25%, indicating that the weighted ensemble more accurately represents the true distribution. The method is compatible with other techniques for improving the performance of particle filters.

The conditioning of least‐squares problems in variational data assimilation

SummaryIn variational data assimilation a least‐squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least‐squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least‐squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian. We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.